Skip to content

fix(v1): transport unbounded harness input safely#2076

Draft
snimu wants to merge 2 commits into
mainfrom
sebastian/subprocess-prompt-stdin
Draft

fix(v1): transport unbounded harness input safely#2076
snimu wants to merge 2 commits into
mainfrom
sebastian/subprocess-prompt-stdin

Conversation

@snimu

@snimu snimu commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add binary stdin support to Runtime.run_program, using native pipes for subprocess and Docker and a runtime-file fallback for remote runtimes
  • reject oversized or NUL-containing argv/environment entries before exec with an actionable error
  • move unbounded task/system/message payloads off argv for Bash, Null, Claude Code, Codex, Pi, Terminus 2, and the Compact example
  • move model-generated Bash and Terminus shell scripts from sh -c <script> to shell stdin
  • keep existing program flags as compatibility inputs for Verifiers-owned harness programs

Motivation

Linux commonly limits each individual exec argument to about 128 KiB, independently of aggregate ARG_MAX. Large task prompts can therefore fail before a harness starts with OSError: [Errno 7] Argument list too long. The same unsafe argv pattern can fail under subprocess, Docker, Modal, or Prime because every backend eventually starts a process.

The general transport policy is:

  1. stdin where the target CLI supports it;
  2. runtime files where stdin is unavailable or a separate payload is required;
  3. argv/environment only for bounded control values, guarded before launch.

Harnesses whose pinned external CLIs do not have verified stdin/file support (Kimi Code, mini-swe-agent, RLM) retain their current launch semantics but now fail early with a clear size error rather than an OS-level E2BIG.

Validation

  • uv run ruff check .
  • uv run ruff format --check .
  • uv run pytest tests/v1/test_configs.py tests/v1/test_taskset.py -q
  • existing Bash and Null subprocess E2E cases
  • manual 200 KB binary/text stdin validation for SubprocessRuntime
  • manual 200 KB Bash and Null payload validation with bounded launch argv
  • ty diagnostics match the current-main baseline

The full local non-Prime/non-Modal matrix could not exercise Docker because Docker Desktop is unavailable on this machine; its Docker-selected failures were infrastructure errors. No tests were added.

@snimu

snimu commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favor of an environment-level design: large paragraph payloads will be written to the rollout workspace and consumed via Bash, so they no longer need to travel in the initial prompt.

@snimu snimu closed this Jul 20, 2026
@snimu snimu changed the title fix(v1): pass bash harness prompts by file fix(v1): transport unbounded harness input safely Jul 20, 2026
@snimu snimu reopened this Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant